Imaging device and imaging method

ABSTRACT

An imaging device of the present invention comprises an image data acquisition circuit for acquiring image data, a temporary storage for temporarily storing the image data, an image processing circuit for subjecting the image data that has been temporarily stored to image processing, a storage medium that can store at least some saved data of the image data that has been temporarily stored, a management information storage for storing management information for managing the save data, and a control circuit for, when the saved data is stored in the storage medium, carrying out management of the saved data using the management information.

Benefit is claimed, under 35 U.S.C. §119, to the filing date of priorJapanese Patent Application No. 2014-049960 filed on Mar. 13, 2014. Thisapplication is expressly incorporated herein by reference. The scope ofthe present invention is not limited to any requirements of the specificembodiments described in the application.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an imaging device and imaging methodfor temporarily storing a plurality of image data at the time of imageprocessing, and carrying out image processing using this temporarilystored image data.

2. Description of the Related Art

With an imaging device such as a digital camera, a plurality of imagesare taken, and the imaging device may be provided with an imageprocessing function for realizing image quality that can not beexpressed with a single image. For example, in Japanese Patent laid-openNo. 2008-271240 (hereafter referred to as patent publication 1) there isproposed an imaging device in which by carrying out image combination bycarrying out positional alignment of a subject using a plurality ofimage data that have been acquired by moving focus position,all-in-focus images and blur-controlled image are generated.

Also, Japanese patent laid-open No. 2011-004353 (hereafter referred toas patent publication 2) proposes HDR (High Dynamic Range) effecttechnology for reproducing gradation characteristics that cannot bereproduced with a single image, by shooting a plurality of images ofdiffering gradation and combining the plurality of images. Japanesepatent laid-open No. 4325625 (hereafter referred to as patentpublication 3) proposes technology for generating an image of higherresolution than an original image by combining a plurality of images.

Also, with an imaging device such as a digital camera, image files arestored in a non-volatile memory such as flash memory. In the event thatdata is stored in flash memory, it is common practice to write data incluster units, and to store in a table (hereafter referred to as FAT(File Allocation Table)) for managing in which cluster data has beenstored.

There is degradation in file rewrite performance due to frequent readingand writing of the FAT and directory information for managing datastored in the flash memory accompanying change in the files. It hastherefore been proposed, in Japanese patent laid-open No. 2005-108304(hereafter referred to as patent publication 4) to prevent reduction inspeed due to degradation using different type of non-volatile memory forstoring management information.

SUMMARY OF THE INVENTION

In the case of combining images, in general the more images that areused in the combination the better the results that are obtained. It istherefore preferable to use as many taken images as possible from theviewpoint of image quality. On the other hand, volatile memory fortemporary storage installed in an image device such as a digital camera(for example, SDRAM (Synchronous Dynamic Random Access Memory)) has alimited capacity that can be installed due to restraints such asmounting pace and product cost, which means that the number of imagesthat can be simultaneously stored is limited.

It has therefore been considered to utilize technology that saves databy temporarily storing as files to a non-volatile storage medium thathas a lower transfer speed than SDRAM but is large capacity, and readsfiles again at the time of required processing. However, if data isstored as files to a non-volatile storage medium, in cases where theimaging device closes down abnormally due to battery depletion etc., orcases where connection or communication between the storage medium andthe imaging device is interrupted, it is not possible to delete filesthat have been temporarily stored, which means they remain as files onthe storage medium, and a region for storing taken images is reduced.

An object of the present invention is to provide an imaging device andan imaging method with which there is no effect on a region for storingtaken images due to temporarily stored files being preserved, even incases where the imaging device has been closed down abnormally or casessuch as where connection or communication between a storage medium andthe imaging device has been severed.

An imaging device of the present invention comprises an image dataacquisition circuit for acquiring image data, a temporary memory fortemporarily storing the image data, an image processing circuit forsubjecting the image data that has been temporarily stored to imageprocessing, a storage medium that can store at least some saved data ofthe image data that has been temporarily stored, a managementinformation memory for storing management information for managing thesaved data, and a control circuit for, when the saved data is stored inthe storage medium, carrying out management of the saved data using themanagement information.

Further, an imaging device of the present invention comprises an imagingcircuit for acquiring image data by forming an image of a subject, amemory for temporarily storing the image data, an image combiningcircuit for combining a plurality of image data and generating combinedimage data, a storage control circuit for controlling data storage, anda control circuit for controlling the imaging device, wherein thecontrol circuit instructs the imaging circuit to consecutively acquire aplurality of image data, instructs the storage control circuit to storeat least some of the plurality of image data using a first storagesystem, instructs the image combination circuit to combine the pluralityof image data, and instructs the storage control circuit to store theimage data that has been combined by the image combining circuit using asecond storage system.

Also, an imaging method of the present invention comprises acquiringimage data, temporarily storing the image data, saving at least some ofthe image data that has been temporarily stored as saved data, storingmanagement information for managing the saved data, and when the saveddata is stored in the storage medium, carrying out management of thesaved data using the management information, and carrying out imageprocessing using the saved data that has been saved.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram mainly showing the electrical structure of acamera of a first embodiment of the present invention.

FIG. 2 is a flowchart showing main operation of the camera of the firstembodiment of the present invention.

FIG. 3 is a flowchart showing main operation of the camera of the firstembodiment of the present invention.

FIG. 4 is a flowchart showing operation of imaging and image processingfor the camera of the first embodiment of the present invention.

FIG. 5 is a flowchart showing operation of imaging and image processingfor the camera of the first embodiment of the present invention.

FIG. 6 is a drawing for describing operation for data save from anSDRAM, in the camera of the first embodiment of the present invention.

FIG. 7 is a drawing for describing a data structure in a storage medium,in the camera of the first embodiment of the present invention.

FIG. 8 is a drawing for describing a data save operation to a storagemedium, in the camera of the first embodiment of the present invention.

FIG. 9 is a block diagram showing the structure of external devices thatcan communicate with the camera of one embodiment of the presentinvention.

FIG. 10 is a flowchart showing operation of imaging and image processingfor a camera of a second embodiment of the present invention.

FIG. 11 is a flowchart showing main operation of external devices forthe second embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

An example where a digital camera is adopted as one embodiment of thepresent invention will be described in the following. This digitalcamera has an imaging section, with a subject image being converted toimage data by this imaging section, and the subject image beingsubjected to live view display on a display section arranged on the rearsurface of the camera body based on this converted image data. Aphotographer determines composition and photo opportunity by looking atthe live view display. At the time of a shutter release operation imagedata is stored in a storage medium. Image data that has been stored inthe storage medium can be subjected to playback display on the displaysection if playback mode is selected.

Also, in summary, the digital camera of the first embodiment can controla FAT for data storage, and, in a memory (SDRAM) for temporary storagewithin the camera, can store at what position (which cluster etc.) in astorage medium save data has been stored. However, the position at whichthe save data has been stored (which cluster etc.) is not stored in thestorage medium. It is therefore possible to identify at what position(which cluster etc.) the save data has been stored from the FAT in thestorage medium, and even if save data remains in the storage medium, indata management it is handled as saved data not being stored, and evenin cases such as where the imaging device closes down abnormally, orcases where communication between the storage medium of the imagingdevice is interrupted, from a practical viewpoint there is nopreservation of saved data (files).

FIG. 1 is a block diagram mainly showing the electrical structure of acamera of a first embodiment of the present invention. This cameracomprises a camera body 100 and an interchangeable lens 200 that can beattached to and removed from the camera body 100. With this embodiment,the photographing lens is an interchangeable lens, but this is notlimiting, and it is also possible to have a digital camera of a typewhere a photographing lens is fixed to the camera body.

The interchangeable lens 200 is made up of a photographing lens 201, adiaphragm 203, a driver 205, a microcomputer 207 and a flash memory 209,and has an interface (hereafter referred to as I/F) 199 between theinterchangeable lens 200 and the camera body 100, which will bedescribed later.

The photographing lens 201 is constituted by a plurality of opticallenses for forming a subject image, and is a fixed focal length lens ora zoom lens. The diaphragm 203 is arranged to the rear on the opticalaxis of this photographing lens 201, and the diaphragm 203 has avariable aperture to control amount of subject light flux passingthrough the photographing lens 201. Also, the photographing lens 201 iscapable of being moved in the optical axis direction by the driver 205,with focus position of the photographing lens 201 being controlled basedon control signals from the microcomputer 207, and in the case where thephotographing lens 201 is a zoom lens, focal length is also controlled.The driver 205 also controls aperture of the diaphragm 203.

The microcomputer 207 that is connected to the driver 205 is connectedto the I/F 199 and the flash memory 209. The microcomputer 207 operatesin accordance with program code stored in the flash memory 209, toperform communication with a microcomputer 121 within the camera body100, which will be described later, and performs control of theinterchangeable lens 200 based on control signals from the microcomputer121.

Besides the previously described program code, various information suchas optical characteristics of the interchangeable lens 200 andadjustment values are stored in the flash memory 209. The I/F 199 is aninterface for carrying out communication between the microcomputer 207inside the interchangeable lens 200 and the microcomputer 121 inside thecamera body 100.

Inside the camera body 100, a mechanical shutter 101 is arranged on theoptical axis of the photographing lens 201. This mechanical shutter 101controls the time for which subject light flux passes, and a well-knownfocal plane shutter or the like is adopted. Behind this mechanicalshutter 101, an image sensor 103 is arranged at a position where thesubject image is formed by the photographing lens 201.

The image sensor 103 functions as an imaging circuit, and hasphotodiodes that constitute each pixel arranged two-dimensionally in amatrix shape, each photodiode generates photoelectric conversion currentin accordance with received light amount, and this photoelectricconversion current is the subject of charge storage by a capacitorconnected to each photodiode. A Bayer array RGB filter is arranged onthe front surface of each pixel. The image sensor 103 also has anelectronic shutter. This electronic shutter carries out control ofexposure time by controlling the time from charge storage until chargereadout of the image sensor 103. The image sensor 103 is not limited toa Bayer array, and a layered type such as Foveon (Registered trademark),for example, can also be used.

The image sensor 103 is connected to an analog processing circuit 105,and this analog processing circuit 105 performs wave shaping on thephotoelectric conversion signals (analog image signals) that have beenread out from the image sensor 103 after reducing reset noise etc., andalso carries out gain increase so as to achieve an appropriatebrightness.

This analog processing circuit 105 is connected to an A/D converter 107,and this A/D converter 107 performs analog to digital conversion on theanalog image signals, and outputs the digital image signals (hereafterreferred to as image data) to a bus 110. In this application, raw imagedata before image processing in the image processing circuit 109 iscalled RAW data.

The A/D converter 107 is connected to a RAW data compression circuit108, and this RAW data compression circuit 108 carries out compressionof RAW data that has been output from the A/D converter 107, and is usedat the time of incorporating RAW data other than for a base image. Abaseimage is an image that is made a base when combining a plurality ofimage data. The RAW data compression circuit 108 converts RAW data thathas been subjected to A/D conversion to data of a smaller data amountthan input data using a polygonal line or a table. Simple lower bittruncation or a conversion method of multiplying by a coefficient ofless than 1 may be used, but taking into consideration gamma conversionfor the image processing circuit 109, conversion using a polygonal lineor a table can reduce image quality degradation.

The bus 110 is a transfer path for transferring various data that hasbeen readout or generated inside the camera body 100, within the camerabody 100. Besides the previously described A/D converter 107 and RAWdata compression circuit 108, an image processing circuit 109, AE (AutoExposure) processing circuit 111, AF (Auto Focus) processing circuit113, image compression circuit 115, image expansion circuit 117, themicrocomputer 121, an SDRAM 127, memory interface (hereafter referred toas memory I/F) 129, display driver 133 and communication circuit 141 areconnected to the bus 110.

The image processing circuit 109 has a RAW data expansion circuit 109 afor expanding RAW data that has been compressed, a basic imageprocessing circuit 109 b for carrying out normal image processing, andan image combining circuit 109 c for carrying out image combining. TheRAW data expansion circuit 109 a, basic image processing circuit 109 band image combining circuit 109 c are used in the case of combining aplurality of images.

Image data other than a base image is compressed by the RAW datacompression circuit 108, and the RAW data expansion circuit 109 acarries out expansion of the compressed RAW data. The RAW data expansioncircuit 109 a applies a reverse characteristic to that of the RAW datacompression circuit 108 and converts to data that is similar toimmediately after A/D conversion. If compression is lossless identicaldata is obtained, while with lossy compression similar data is convertedto. Generally, data after 18 conversion has a linear characteristic withrespect to exposure amount, and so compressed RAW data stored in theSDRAM is converted to linear RAW data.

The basic image processing circuit 109 a performs optical black (OB)subtraction processing, white balance (WB) correction, demosaicingprocessing carried out in the case of Bayer data, color reproductionprocessing, gamma correction processing, color matrix computation, noisereduction (NR) processing, edge enhancement processing etc. on RAW data.In a single exposure, and if a special effect filter has not been set,image processing is completed with only processing by this basic imageprocessing circuit 109 a.

The image combining circuit 109 c carries out various image combinationin accordance with a combining mode that has been set etc. With thisembodiment, it is possible to set three combination modes, namely HDRcombination, focus stacking and super resolution combination. In theevent that HDR combination mode has been set, the image combiningcircuit 109 c generates an image having a wider dynamic range than asingle image by carrying out combination of a plurality of image datathat have been taken at a plurality of exposure amounts with positionalalignment to a base image.

Also, in the event that focus stacking mode has been set, the imagecombining circuit 109 c generates an image having a depth of field thatis different to a single image, by carrying out combination of aplurality of image data that have been taken at a plurality of focuspositions, with positional alignment to abase image. In the event thatsuper resolution combination mode has been set, the image combiningcircuit 109 c generates an image having a better resolution than asingle image by carrying out combination of a plurality of image datawith positional alignment to a base image.

The AE processing circuit 111 measures subject brightness based on imagedata that has been input via the bus 110, and outputs this subjectbrightness information to the microcomputer 121 via the bus 110. Adedicated photometric sensor for subject brightness measurement may beprovided, but in this embodiment subject brightness is calculated basedon image data.

The AF processing circuit 113 extracts signals for high frequencycomponents from image data, acquires a focus evaluation value usingintegration processing, and outputs the focus evaluation value via thebus 110 to the microcomputer 121. In this embodiment, focusing of thephotographing lens 201 is carried out using the so-called contrastmethod. With this embodiment AF control using a contrast method is givenas an example, but it is also possible to carry out focusing by AFcontrol using phase difference AF, where subject light flux is dividedand phase difference sensors are provided on the optical paths, or aphase difference sensor is provided on an image sensor.

At the time of storage of image data to the storage medium 131, theimage compression circuit 115 subjects image data that has been read outfrom the SDRAM 127 to compression in accordance with various compressionformats such as JPEG compression in the case of a still picture or MPEGin the case of a movie.

The image expansion circuit 117 also carries out expansion of JPEG imagedata and MPEG image data for image playback display. In the expansion, afile that is stored in the storage medium 131 is read out, and afterbeing subjected to expansion processing in the image expansion circuit117 the expanded image data is temporarily stored in the SDRAM 127. Withthis embodiment, the JPEG compression system and/or MPEG compressionsystem are adopted as the image compression system, but the compressionsystem is not limited to this and another compression systems may beused, such as TIFF, H.264 etc. The compression system may also belossless compression or lossy compression.

The microcomputer 121 provides a function as a control circuit for thisentire camera, and performs overall control of various sequences of thecamera in accordance with program code stored in the flash memory 125.Besides the previously described I/F 199 an operation member 123 and aflash memory 125 are connected to the microcomputer 121.

The operation member 123 includes operation members such as variousinput buttons, like a power supply button, shutter release button, amovie button, playback button, menu button, cross-shaped key, and OKbutton, and various input keys, and detects operating states of theseoperation members and outputs the result of detection to themicrocomputer 121. The microcomputer 121 executes various sequencesaccording to user operation based on the result of detection from theoperation member 123. The power supply button is an operation member forinstructing to turn a power supply of the digital camera on or off. Ifthe power supply button is pressed, the power supply of the digitalcamera is turned on, and if the power supply button is pressed onceagain the power supply of the digital camera is turned off.

The shutter release button has two switches. Specifically, the shutterrelease button has a first release switch that turns on when the buttonis pressed down half way, and a second release switch that is turned onwhen the button is pressed down further from the half-pressed state tobe pressed down fully. The microcomputer 121 executes shootingpreparation sequences such as an AE operation and AF operation if thefirst release switch is turned on. Also, if the second release switch isturned on shooting is carried out by executing a series of shootingsequences to control the mechanical shutter 101 etc., acquire image databased on a subject image from the image sensor 103 etc., and store thisimage data in the storage medium 131.

The movie button is an operation button for designating start or finishof movie shooting, and if the movie button is initially operated movieshooting commences, and movie shooting finishes when it is operatedagain. The playback button is an operation button for setting andcanceling playback mode settings, and if playback mode is set image dataof a taken image is read out from the storage medium 131, and a takenimage is playback displayed on the display panel 135.

The menu button is an operation button for causing display of menuscreens on the display panel 135. It is possible to carry out thevarious camera settings on the menu screens. As camera settings thereare, for example, combination modes such as HDR combination, focusstacking and super resolution combination.

The flash memory 125 is a non-volatile memory, and stores program codefor executing various sequences for the microcomputer 121, andparameters for image processing such as white balance gain and colormatrix etc. The microcomputer 121 carries out overall control of thecamera based on this program code.

The SDRAM 127 is a volatile memory that can be electrically rewritten,for temporary storage of image data. This SDRAM 127 temporarily storesimage data that has been output from the A/D converter 107, and imagedata that has been processed in the image processing circuit 109, imagecompression circuit 115 and image expansion circuit 117 etc.

The memory I/F 129 is connected to the storage medium 131, and carriesout control for reading and writing of data, such as image data andheader information attached to image data, to and from the storagemedium 131. The storage medium 131 is a storage medium such as a memorycard that can be loaded into and taken out of the camera body 100, butthis is not limiting and it may also be a hard disk or the like builtinto the camera body 100.

The display driver 133 is connected to the display panel 135, and readsout from the SDRAM 127 and storage medium 131 to display an image on thedisplay panel 135 based on image data that has been expanded by theimage expansion circuit 117. The display panel 135 is arranged on a rearsurface of the camera body 100, and carries out image display. Since thedisplay panel 135 is arranged on a display surface of an externalsection of the camera, such as the rear surface, it is a display sectionthat is prone to the effects of external light, but it is possible toinstall a large-sized display panel. As a display section it is possibleto adopt various display panels such as a liquid crystal display panel(LCD, TFT), or organic EL etc.

As image display modes of the display panel 135, there are QuickviewDisplay for displaying image data to be stored for only a short timeimmediately after being taken, playback display of image files for stillimages and movies that have been stored in the storage medium 131, andmovie display such as live view display.

The communication circuit 141 carries out communication with externaldevices by wired communication using USB or LAN etc., or wirelesscommunication, and transmits live view images, or playback images thathave been read out from the storage medium 131, to an external displaysection such as a smart phone or television. The communication circuit141 also carries out exchange of some or all of temporarily stored imagedata between storage media of external devices (such as, for example,memory of a PC, smartphone or server on a network) when carrying outimage composition in the image combining circuit 109 c using a pluralityof images.

Next, processing of a main flow of the camera of this embodiment will bedescribed using the flowcharts shown in FIG. 2 and FIG. 3. Theflowcharts shown in FIG. 2, FIG. 3, and in FIG. 4, FIG. 5 and FIG. 10,which will be described later, are executed by the microcomputer 121controlling each section in accordance with program code stored in theflash memory 125.

If the power supply button within the operation member 123 is operatedand the power supply is turned on, the microcomputer 121 commencesoperation in accordance with the main flow shown in FIG. 2. Onceoperation has commenced, initialization is first executed (S1). As partof the initialization, mechanical initialization and electricalinitialization, such as initialization of various flags etc., arecarried out. As one of the various flags, a storage in progress flagindicating whether or not movie storage is in progress is reset to off(refer to steps S13, S15 and S31 etc.).

Once initialization has been carried out, it is next determined whetheror not the playback button has been pressed (S3). Here, determination iscarried out by detecting the operating state of the playback buttonwithin the operation member 123. If the result of this determination isthat the playback button has been pressed, playback/editing mode isexecuted (S5). Here, image data is read out from the storage medium 131,and a table of still images and movies is displayed on the LCD 135. Theuser selects an image from within the table by operating thecross-shaped key and decides on the image using the OK button. It isalso possible to carry out editing of a selected image.

If playback/editing is executed in step S5, or if the result ofdetermination in step S3 was that the playback button had not beenpressed, it is determined whether or not camera setting will be carriedout (S7). When the menu button within the operation member 123 has beenoperated, camera setting is carried out on a menu screen. In this stepit is therefore determined whether or not this menu button has beenpressed.

If the result of determination in step S7 is that the menu button hasbeen pressed, camera setting is carried out (S9). As describedpreviously, it is possible to carry out various camera settings on themenu screens. As camera setting, as was described previously, it ispossible to set modes, such as normal shooting, HDR combination, focusstacking, or super resolution combination, as shooting modes. It is alsopossible to set modes such as JPEG storage, TIFF storage, JPEG-RAWstorage or RAW storage as still picture storage modes. It is possible toset modes such as motion JPEG storage or H.264 storage as the moviestorage modes. It is possible to set modes such as fine or normal asimage quality modes.

If camera setting has been carried out in step S9, or if the result ofdetermination in step S7 was not camera setting, it is next determinedwhether or not the movie button has been pressed (S11). Here themicrocomputer 121 carries out determination based on operating state ofthe movie button input from the operation member 123.

If the result of determination in step S11 is that the movie button hasbeen pressed, inversion of the storage in progress flag is next carriedout (S13). The storage in progress flag is set to on (1) if movieshooting is progress, or set to off (0) if a movie is not being shot. Inthis step the flag is inverted, that is, if it is set to on (1) it isinverted to off (0), and if it is set to off (0) it is inverted to on(1).

Once inversion of the storage in progress flag has been carried out instep S13, it is next determined whether or not storage of a movie is inprogress (S15). Here, determination is based on whether the storage inprogress flag, which was inverted in step S13, is set to on or set tooff.

If the result of determination in step S15 is that movie storage is inprogress, a movie file is created (S19). Movie storage is carried out instep S61, which will be described later, but in this step a movie filefor movie storage is created, and preparation so as to be able to storeimage data of the movie is performed.

On the other hand, if the result of determination is that move storageis not in progress, the movie file is closed (S17). The movie file isclosed in this step because the movie button has been pressed and movieshooting is completed. When closing the movie file, the movie file isplaced in a state of being capable of being played back by storing anumber of frames in a header of the movie file etc., and file writing iscompleted.

Once the movie file has been closed in step S17, or if a movie file iscreated in step S19, or if the result of determination in step S11 wasthat the movie button was not pressed, it is next determined whether ornot movie storage is in progress (S31). In this step, similarly to stepS15, determination is based on whether the storage in progress flag ison or off.

If the result of determination in step S31 is that movie storage is notin progress, it is next determined whether or not the shutter releasebutton has been pressed down half way, in other words, whether or notthe first release switch has changed from an off state to an on state(S33). For this determination, the state of the first release switchthat is linked to the shutter release button is detected by theoperation member 123, and determination is carried out based on theresult of this detection. If the result of detection is that the firstrelease switch has changed from an off state to an on state, the resultof determination becomes Yes, while on the other hand if an on state oran off state is maintained the result of determination becomes No.

If the result of determination in step S33 is that the shutter releasebutton has been pressed down half way, then in the case where the firstrelease has transitioned from the off state to the on state an AE/AFoperation is executed (S35). Here, the AE processing circuit 111 detectssubject brightness based on image data acquired by the image sensor 103,and calculates shutter speed and aperture value etc. for correctexposure based on this subject brightness.

Also, an AF operation is carried out in step S35. Here, the driver 205moves focus position of the photographing lens 201 by means of themicrocomputer 207 inside the interchangeable lens 200, so that a focusevaluation value that has been acquired by the AF processing circuit 113becomes a peak value . As a result, if the shutter release button ispressed down halfway when movie shooting is not being carried out,focusing of the photographing lens 201 is carried out at that point intime. After that processing advances to step S37.

If the result of determination in step S31 is that there has not been atransition from the shutter release button being off to the firstrelease switch being in an on state, it is next determined whether ornot the shutter release button has been pressed down fully, and thesecond release switch is in an on state (S41). In this step, the stateof the second release switch that is linked to the shutter releasebutton is detected by the operation member 123, and determination iscarried out based on this result of detection.

If the result of determination in step S41 is that the shutter releasebutton is pressed down fully and that the second release switch hasentered an on state, shooting and image processing are carried out(S43). Here, the diaphragm 203 is controlled with the aperture valuethat was calculated in step S35, and the shutter speed of the mechanicalshutter 101 is controlled with the calculated shutter speed. Then, oncethe exposure time corresponding to the shutter speed has elapsed, imagesignals are read from the image sensor 103, and RAW data that has beenprocessed by the analog processing circuit 105 and the A/D converter 107is output to the bus 110.

Also, if shooting is carried out in step S43, image processing iscarried out. RAW data that has been acquired by the image sensor 103 isread out, and image processing is carried out by the image processingcircuit 109. Also, in a case where an image combination mode has beenset, shooting is carried out a plurality of times in accordance with theimage combination mode, and image data to be used at the time of imagecombination is temporarily saved to the storage medium 133 etc. Detailedoperation of this shooting and image processing will be described laterusing FIG. 4 and FIG. 5.

Once shooting and image processing have been carried out, still picturestorage is next carried out (S45). Here, image data for a still pictureto which image processing has been applied is stored in the storagemedium 131. At the time of still picture storage, storage is carried outin a format that has been set. In the event that JPEG has been set, datathat has already been subjected to image processing is subjected to JPEGcompression by the image compression circuit 115 and stored. Also, inthe case of TIFF format, data is converted to RGB data and stored in RGBformat. In the case where RAW storage has been set, if combination hasbeen carried out with RAW data that was acquired by shooting, combinedRAW data is also stored. A storage destination for the image data may bestorage medium 131 within the camera body, or may be storage to anexternal device via the communication circuit 14.

If the result of determination in step S41 is not that the secondrelease button is not fully pressed, or if the result of determinationin step S31 is that movie storage is in progress, next AE is carried out(S51). In the case where determination in previously described step S41was No, it is a situation where no operation was carried out for theshutter release button, and in this case live view display of step S57,which will described later, is carried out. Also, if the determinationin previously described step S31 was Yes, movie storage is in progress.In this step, shutter speed and ISO sensitivity for the electronicshutter of the image sensor 103 in order to carry out live view displayor movie shooting at the correct exposure are calculated.

Once AE has been carried out, shooting using the electronic shutter iscarried out (S53). Here, the subject image is converted to image data.Specifically, charge storage is performed during an exposure time thatis determined by the electronic shutter of the image sensor 103, andimage data is acquired by reading out the stored charge once theexposure time has elapsed.

Once shooting using the electronic shutter has been carried out, imageprocessing is carried out on the acquired image data (S55). In thisstep, basic image processing, such as WB correction, color matrixcalculation, gamma conversion, edge enhancement, and noise reduction arecarried out by the basic image processing circuit 109 b.

If basic image processing has been carried out, live view display isnext carried out (S57). In this step, live view display on a displaypanel 135 is carried out using the image data that was subjected tobasic image processing in step S55. Specifically, image data is acquiredin step S53, and image processing is carried out, and so update of thelive view display is carried out using an image that has been subjectedto this processing. A photographer can determine composition and photoopportunity by looking at the live view display.

Once live view display has been carried out in step S57, it is nextdetermined whether or not movie storage is in progress (S59). Here it isdetermined whether or not the storage in progress flag is on. If theresult of this determination is that movie storage is in progress, moviestorage is carried out (S61). Here, image data read out from the imagesensor 103 is subjected to image processing to give image data for amovie, and stored in a movie file.

If movie storage has been carried out in step S61, if the result ofdetermination in step S59 was not that movie storage is in progress, ifstill picture storage has been carried out in step S45, or if AE/AF wascarried out in step S35, it is next determined whether or not the powersupply is off (S37). In this step it is determined whether or not thepower supply button of the operation member 123 has been pressed again.If the result of this determination is not power off, processing returnsto step S3. On the other hand, if the result of determination is poweroff, a termination operation of the main flow is executed and then thismain flow is terminated.

In this way, in the main flow of the first embodiment of the presentinvention, it is possible to set an image combination mode etc. (S9),and if an image combination mode has been set shooting is carried out aplurality of times depending on the image combination mode in shootingand image processing (S43), and the combined image data is stored (S45).At the time of image combination, at least some of the image data canbeing temporarily saved to the storage medium 131 etc.

Next, detailed operation of the shooting and image processing in stepS43 will be described using the flowcharts shown in FIG. 4 and FIG. 5.If the flow for shooting and image processing is entered, firstappropriate exposure shooting is carried out (S101). Here, shooting iscarried out at the exposure conditions that have been calculated in stepS35 at the time the shutter release button was pressed down half way.Depending on the shooting mode etc., shooting in this step need not beat an appropriate exposure.

Once appropriate exposure shooting has been carried out, next RAWincorporation is carried out (S103). Shooting in step S101 is a baseimage, and in order to prioritize image quality data compression is notcarried out by the RAW data compression circuit 108 and insteadnon-compressed RAW image data is incorporated and stored in the SDRAM127.

Once RAW the incorporation has been carried out, it is next determinedwhether or not there is HDR combination (S105). It is determined whetheror not HDR combination mode has been set as the shooting mode in thecamera setting of step S9. HDR combination mode is a mode for generatingan image of wider dynamic range than a single image, using a pluralityof image data that have been acquired by sequential shooting atdifferent exposure amounts.

If the result of determination in step S105 is HDR combination, exposureamount change is carried out (S111). Exposure amount is sequentiallychanged to, for example, −1 step (an exposure amount of half compared tothe appropriate exposure conditions), and +1 step (an exposure amount ofdouble compared to the appropriate exposure condition) with respect toappropriate exposure conditions that were determined in step S35. Oncethe exposure amount has been changed, shooting is carried out (S113).Here, shooting is carried out in accordance with exposure amount thatwas changed in step S111.

Once shooting has been carried out, RAW compression and incorporation iscarried out (S115). An image that has been shot with exposure amountchanged is not a base image, and so a certain amount of degradation inimage quality is permissible. Therefore, in order to make image datavolume small, image data that has been read out is subjected to datacompression by the RAW data compression circuit 108.

Once Raw compression and incorporation has been carried out, is nextdetermined whether or not shooting has been carried out a specifiednumber of times (S117). Here, it is determined whether or not a numberof images that has been set for carrying out HDR combination, forexample, a number of taken images besides the base image, has reached 2.If the result of this determination is that the specified number ofimages has not been reached, step S111 is returned to and shooting forHDR combination continues.

If the result of determination in step S105 is not HDR combination, itis next determined whether or not it is focus stacking (S121). It isdetermined whether or not focus stacking mode has been set as theshooting mode in the camera setting of step S9. As was describedpreviously, focus stacking mode is a mode for generating an image havinga different depth of field to a single image, using a plurality of imagedata that have been acquired by sequentially shooting by moving thefocus lens to different positions.

If the result of determination in step S121 is focus stacking, moving ofthe focus lens is carried out (S123). Here, instructions so as to causethe photographing lens 201 to be sequentially moved to specified focuspositions are sent to the microcomputer 207 within the interchangeablelens 200. As specified positions, for example, it is possible to havepositions that are changed ⅛th of a step at a time from the close-up endto the infinity end.

Once focused movement has been carried out, next appropriate exposureshooting is carried out (S125). Here, shooting is carried out by settingexposure amount in accordance with appropriate exposure conditions thatwere determined in step S35. Once optimal exposure shooting has beencarried out, next, similarly to step S115, RAW compression andincorporation are carried out (S127). Here, image data that has beenreadout is subjected to data compression by the RAW data compressioncircuit 108.

Once Raw compression and incorporation has been carried out, it is nextdetermined whether or not shooting has been carried out a specifiednumber of times (S129). Here, it is determined whether or not there area number of images set for carrying out focus stacking, for example, inthe event that a number of steps for focus position has been set to ⅛,for example, it is determined whether or not a number of taken imagesbesides a base image has reached 8. If the result of this determinationis that the specified number of images has not been reached, step S123is returned to and shooting for focus stacking continues.

If the result of determination in step S121 is not focus stacking, it isnext determined whether or not it is super resolution combination(S131). It is determined whether or not super resolution combinationmode has been set as the shooting mode in the camera setting of step S9.Super resolution combination mode is a mode for generating an image thathas better resolution than a single photograph using a plurality ofimage data. Specifically, sequential shooting of a plurality of imagesis carried out while shifting a sensor (image sensor 103), in the caseof sensor vibration correction, or a correction lens, in the case oflens vibration correction, by a specified number of pixels (for example,by a single pixel of the sensor). Once a plurality of image data havebeen acquired, an image having high resolution is generated that isdifferent to that of a single image, using these image data.

If the result of determination in step S131 is super resolutioncombination, minute optical axis movement is carried out (S133). Here,in the case of sensor vibration correction, the sensor position is movedminutely within a plane orthogonal to the optical axis, so that theimage sensor 103, for example, moves by one pixel. Also, in the case oflens vibration correction, the state of a lens for correction isminutely corrected so that, for example, the optical axis is moved byone pixel on the sensor.

Once minute optical axis movement has been carried out, then, similarlyto step S125, appropriate exposure shooting is carried out (S135). Here,shooting is carried out in accordance with appropriate exposureconditions that were determined in step S35. Once appropriate exposureshooting has been carried out, next, similarly to step S115, RAWcompression and incorporation are carried out (S137). Here, image datathat has been read out is subjected to data compression by the RAW datacompression circuit 108.

Once Raw compression and incorporation has been carried out, it is nextdetermined whether or not shooting has been carried out a specifiednumber of times (S139). Here, a specified number of images set forcarrying out super resolution combination is, for example, three imagesbesides the base image, namely images that have been offset from theposition where the base image was taken to the left by one pixel, belowby one pixel, and to the lower left by one pixel, and it is determinedwhether or not three images have been taken. If the result of thisdetermination is that the specified number of images has not beenreached, step S133 is returned to and shooting for super resolutioncombination continues.

If the result of determination in S117, S129 or S139 is that thepredetermined number of pixels has been reached, next data save iscarried out (S141). Raw data that has been read out from the imagesensor 103 is stored in the SDRAM 127, either directly, with nocompression, or after being compressed. The image combining circuit 109c carries out image combination such as HDR combination using aplurality of image data (RAW data) stored in the SDRAM 127, but at thistime, in order to secure a memory region required for image combinationin the SDRAM 127, at least some of the image data (RAW data) stored inthe SDRAM 127 is saved to the storage medium 131. Detailed operation ofthis data save will be described later using FIG. 7 and FIG. 8.

Once data save has been carried out, image combination is carried out instep S151 and after. First, similarly to step S105, whether or not it isHDR combination is determined (S151). If the result of thisdetermination is HDR combination, RAW acquisition and expansion arecarried out (S153). RAW data for a base image and RAW data forcombination images is obtained either from the SDRAM 127 or from a savedestination of the storage medium 131. In this case, RAW data for thebase image is not compressed, but RAW data for the combination images iscompressed, and so expansion processing is carried out on the RAW datafor the combination images in the RAW data expansion circuit 109 a. Theacquired RAW data for the base image and the expanded RAW data for thecombination images is stored in a working region of the SDRAM 127.

Once RAW the acquisition and expansion have been carried out, nextalignment is carried out (S155). Here, RAW data for the base image andRAW data for the combination images that were acquired instead S153 orread out from the SDRAM 127, and the image combining circuit 109 ccarries out section by section alignment of the read-out base image andcombination images.

Once alignment has been carried out, next HDR combination is carried out(S157). Here, the image combining circuit 109 c combines RAW data forcombination that has been sequentially shot at different exposureamounts with RAW data for the base image, or, if this is the second orlater combination with this repeated processing, with the previouscombination result, to generate an image having a wider dynamic rangethan a single image.

Once HDR combination has been carried out, next, it is determinedwhether or not combination of a specified number of images has beencarried out (S159). Here, it is determined whether or not HDRcombination has been carried out for a number of combination images,that is, a number of images that have been taken for HDR combination. Ifthe result of this determination is that the specified number of imageshas not been reached, step S153 is returned to and HDR combinationcontinues.

If the result of determination in step S151 was not HDR combination,next, similarly to step S121, it is determined whether or not it isfocus stacking (S161). If the result of this determination is focusstacking, RAW acquisition and expansion are carried out (S163),similarly to step S153. Next alignment is carried out similarly to stepS155 (S165).

Once alignment has been carried out, next focus stacking is carried out(S167). Here, RAW data for combination that has been sequentially shotat different focus positions is combined with RAW data for the baseimage, or, if this is the second or later combination with this repeatedprocessing, with the previous combination result, to generate an imagehaving a different depth of field to a single image.

Once focus stacking has been carried out, next, it is determined whetheror not combination of a specified number of images has been carried out(S169). Here, it is determined whether or not focus stacking has beencarried out for a number of combination images, that is, a number ofimages that have been taken for focus stacking. If the result of thisdetermination is that the specified number of images has not beenreached, step S163 is returned to and focus stacking continues.

If the result of determination in step S161 was not focus stacking,next, similarly to step S131, it is determined whether or not it issuper resolution combination (S171). If the result of this determinationis super resolution combination, RAW acquisition and expansion arecarried out (S173), similarly to step S153. Next alignment is carriedout (S175) similarly to step S155.

Once alignment has been carried out, next super resolution combinationis carried out (S177). Here, RAW data for combination that has beensequentially shot at different positions on the optical axis is combinedwith RAW data for the base image, or, if this is the second or latercombination with this repeated processing, with the previous combinationresult, to generate an image having a different resolution to a singleimage.

Once super resolution combination has been carried out, next, it isdetermined whether or not combination has been carried out for aspecified number of images (S179). Here, it is determined whether or notsuper resolution combination has been carried out for a number ofcombination images, that is, a number of images that have been taken forsuper resolution combination. If the result of this determination isthat the specified number of images has not been reached, step S173 isreturned to and super resolution combination continues.

If the result of determination in steps S159, S169 or S179 is thespecified number of images, basic image processing is carried out(S181). Here, the basic image processing circuit 109 b carries out basicimage processing, specifically image processing such as WB correction,color matrix computation, gamma conversion, edge enhancement, noisereduction etc., on RAW data that was combined in steps S157, S167 andS177. Once basic image processing has been carried out, the shooting andimage processing flow is completed, and the originating processing flowis returned to. Various image processing parameters, such as whitebalance gain, color matrix and gamma etc. used in the basic imageprocessing may be changed for each combination mode.

In this manner, in the flow for shooting and image processing of thisembodiment, if shooting is carried out in line with a set shooting mode(combination mode) (S101-S139), RAW data is temporarily saved to thestorage medium 131. As a result, a memory region for image combinationis secured in the SDRAM 127, and image combination is carried out usingthis secured memory region. It is therefor possible to acquire manyimages for image combination, and it is possible to improve imagequality of the combined image.

With this embodiment, image data for combination is stored in the SDRAM127 after having been compressed in the RAW data compression circuit 108(S115, S127 or S137), and expanded in the RAW data expansion circuit 109a at the time of image combination (S153, S163 or S173). However, imagedata for combination is also stored uncompressed in the SDRAM 127, andmay be used directly at the time of image combination. In this case, agreater memory capacity is required, but it is possible to shorten thetime required for compression and expansion.

Next, data save will be described using FIG. 6 to FIG. 8. FIG. 6explains storage states for data within the SDRAM 127. In FIG. 6,program regions etc. have been omitted, and only data portions areshown.

In FIG. 6 state 6 a shows a data storage state for a data region beforeshooting, with RAW data not being stored. State 6 b shows a storagestate for RAW data within the SDRAM 127 after shooting. With the exampleshown in state 6 b, a total of 8 data items are stored, namely base RAWdata, and RAW data for combination A to RAW data for combination G. Atthe time of shooting (S113, S125 and S135 in FIG. 4), completingshooting of images for combination in as short a time as possible makesit possible to prevent a combination failure due to subject blur etc.

State 6 c shows a state where some of the RAW data stored in the SDRAM127 has been subjected to data save in the storage medium 131.Specifically, by data saving the RAW data for combination D-G that hasbeen stored in the SDRAM 127 to the storage medium 131, a working region127 a required for combination is freed up. If the working region 127 ais freed up, the image combining circuit 109 c executes combinationprocessing, in accordance with a shooting mode (combination mode) thathas been set, in the working region 127 a. Saving of RAW data forcombination may be carried out in parallel with processing for shootingthe RAW data for combination.

State 6 d shows a state where image combination has been carried out bythe image combining circuit 109 c using base RAW data and RAW data forcombination A-C. In this state, combined RAW data that has beengenerated using the base RAW data and the RAW data for combination A-Care stored in the region where the base RAW data was stored.

State 6 e shows a state where RAW data for combination D that had beensaved to the storage medium 131 is returned to an empty region of theSDRAM 127. In this state, the image combining circuit 109 c carries outcombination processing for combined RAW data and RAW data forcombination D, and replaces the combined RAW data after combination inthe working region 127 a. Returning the RAW data for combination thathad been saved in the storage medium 131, and the combinationprocessing, may be processed in time sequence, or they may both beprocessed in parallel. Also, with the example shown in state diagram 6e, only the RAW data for combination D has been returned to the emptyregion, but is also possible to return a plurality of RAW data forcombination, such as RAW data for combination E, F etc., and not justcombined RAW data D.

State 6 f shows a state where image combination has been completed.Specifically, in state 6 e, once image combination has been repeated bysequentially returning RAW data for combination from the storage medium131 to the SDRAM 127 and image combination has been carried out usingall RAW data for combination, combination complete RAW data isgenerated, as shown in state 6 f.

FIG. 7 is a drawing showing data storage regions of the storage medium131. As shown in FIG. 7, as storage regions of the storage medium 131there are a file allocation table (FAT) region 131 a, a directory entrystorage region 131 b and a data storage region 131 c.

The file allocation table (FAT) region 131 a is a region for managing bystoring in which order one or more clusters is being used. Also, thedirectory entry storage region 131 b is a region showing what type offile has been stored. The data storage region 131 c is a region forstorage of data, with data being stored for each cluster.

Data indicating how a cluster, which is a unit for storage, is used fora data storage region is stored in the FAT. Since the maximum capacityfor storage in a single cluster is set, if data cannot be stored in asingle cluster an empty cluster is searched for, and informationindicating that storage is continuing to the empty cluster is stored inthe FAT. In the event that there is no continuation to other clusters, anumerical value indicating that there is no continuation is stored.Cluster association information indicating which cluster data uses iscalled a cluster chain. A header of a cluster chain can be identified bya header cluster number for a file of a directory entry.

When data is stored in the data region 131 c, unused clusters aredetected by searching within the FAT and data is written. If data amountis large and cannot all be written to that cluster, the next emptycluster is searched for, the fact that data continues to the emptycluster is stored in the FAT, and data is written to the empty cluster.If this processing is repeated and it is possible to write all of thedata, data indicating that no further clusters are joined is written tothe FAT (END in FIG. 8). After that, file name information, data size,and head cluster number are stored in the directory entry.

When data is read from the data storage region 131 c, size and headcluster is acquired from the file name stored in the directory entrystorage region 131 b. If the head cluster is known, data correspondingto the head cluster is read out from the data storage region 131 c, andif the size has not been reached reading is repeated by acquiring thenext cluster from the FAT.

Generally, because it takes time to write to the storage medium, data ofthe FAT storage region and the directory entry storage region istemporarily read to SDRAM within a camera, then necessary editing iscarried out and data written to the storage medium.

With this embodiment, as shown in FIG. 8, when data to be saved iswritten in the storage medium, a cluster chain is stored in a FAT 127 t(stored in SDRAM 127) within the camera, similarly to the related art,but in addition to this storage, management information for identifyinga cluster chain for save data is stored. With the example shown in FIG.8, saved data is stored in cluster numbers 9-11 of the data storageregion of the storage medium 131, and as management information “1” isstored as a mask bit 127 u. “0” is stored as a mask bit 127 u inclusters in which saved data is not stored.

Also, cluster number where data is stored, file name, and size arestored in directory entry 127 v within the SDRAM 127. However, data thathas been saved is stored to be managed not in a directory entry but in asaved data entry region 127 w. In the storage medium 131, the FAT storesa cluster, in which save data is stored, in directory entry 131 v as anempty cluster, and does not store an entry for saved data.

When reading out data saved in the storage medium 131, a head cluster(cluster 9 in the example of FIG. 8) in which necessary data is storedis acquired from an entry 127 w for saved data, and data up to END isread from the storage medium 131 while referencing the FAT 127 t withinSDRAM. With the example shown in FIG. 8, data stored in clusters 9, 10and 11 is read.

In this way, management information for the saved data is stored in theSDRAM 127, but management information is not stored in the storagemedium 131 even though data is saved. As a result, a file in which saveddata has been stored appears to no longer exist according to themanagement information for the storage medium 131. Specifically,information relating to a file in which save data has been stored is notstored in the directory 131 v or the FAT 131 t of the storage medium131, which means that it is possible to avoid a problem where files thatthe user is unaware of remain in the storage medium 131, even if thereis abnormal close down (the camera battery running out or the storagemedium being removed etc.).

In this embodiment, a plurality of image data are temporarily stored inSDRAM 127 by carrying out shooting a plurality of times (S43 in FIG. 3,S101-S139 in FIG. 4), and when carrying out image combination using thetemporarily saved plurality of image data (S151-S179 in FIG. 5), atleast some of the plurality of image data are saved to storage medium131 (S141 in FIG. 5), and the saved data are returned to SDRAM 127 andused at the time of image combination (refer to S153, S163 and S173 inFIG. 5, and state 6 e in FIG. 6). As a result, it is possible to performimage combination using a lot of image data even if capacity of theSDRAM 127 is small, and it is possible to improve image quality of acombined image.

Also, when saving image data to the storage medium 131, managementinformation for save (portions of the saved data entry 127 w and the FAT127 t having 1 set in a mask bit 127 u) is stored within the SDRAM 127,but not stored in the storage medium 131. As a result, even in the casewhere the camera has been closed down abnormally, and saved data hasbeen returned to SDRAM 127, since management information for saved datais not stored in the storage medium 131, saved data is not actuallypreserved.

Next, a second embodiment of the present invention will be describedusing FIG. 9 to FIG. 11. In the first invention, since direct access ispossible to the file system of the storage medium 131 within the camera,it was possible to control management information for saved data (saveddata entry 127 w and FAT 127 t, and information of the mask bits 127 u).Conversely, with the second embodiment, the storage medium is anexternal data storage device connected via the communication circuit 14.For a device that is external to the camera, the camera cannot directlyaccess files, and it is not possible for the camera body to controlmanagement information for saved data. Therefore, if a connectionbetween the camera and an external data storage device is lost, saveddata will be automatically deleted after a fixed time. Specifically,data receipt, transfer, and file deletion are executed at the externaldata storage device side.

The structure at the camera side of this embodiment is the same as thecamera of the first embodiment that was shown in FIG. 1. Also, theexternal data storage device is in a PC, smartphone, network storageetc. One example of the structure of an external data storage devicewill be described using FIG. 9.

The external storage device 300 comprises a communication circuit 301,memory 303, a control circuit 305, and a user interface 307. Thecommunication circuit 301 can carry out exchange of data with thecommunication circuit 14 within the camera body 100 using wired orwireless communication.

The memory 303 is an electrically rewritable volatile memory ornon-volatile memory, and can store image data that has been input viathe communication circuit 301. Program code etc. for controlling theexternal data storage device is also stored. This memory 303 has thesame storage regions as the storage medium 131 was shown in FIG. 7 andFIG. 8.

The control circuit 305 carries out overall control of the externalstorage device 300 in accordance with program code stored in the memory303. The user interface 307 has a display section and an operatingsection, and displays information to the user as well as being inputwith instructions from the user. The external storage device 300 issimply memory, and the user interface 307 need not be provided as longas control by means of communication etc. is possible.

Next, operation of this embodiment will be described using FIG. 10 andFIG. 11. Main operation of the camera is similar to the flowcharts shownin FIG. 2 and FIG. 3, and so detailed description is omitted. However,in step S45 of FIG. 3 still picture image data to be stored in theexternal data storage device 300 which will be described later istransmitted without designating a deletion time, which will be describedlater, and stored in the external data storage device 300 as a file in aformat that will not be deleted after the elapse of a specified time.Also, operation for shooting and image processing in step S43 of FIG. 3is different in that within the flowcharts shown in FIG. 4 and FIG. 5,FIG. 5 is replaced with FIG. 10 (the flowchart shown in FIG. 4 is thesame). Description will therefore concentrate on points of difference,using FIG. 10.

The flowchart shown in FIG. 10 only has steps S154, S164 and S174 addedto the flowchart shown in FIG. 5, and other steps are the same as in theflowchart of FIG. 5. Description will therefore concentrate on points ofdifference.

In this embodiment, if the operation for shooting and image processingis commenced, first, in S101-S139 in FIG. 4, shooting is carried out anda plurality of image data are acquired. Once the plurality of image datahave been acquired, data save is carried out (S141). Here, temporarydata is saved to memory 303 within the external data storage device 300,and a memory region required for combination is secured. Specifically,data is saved to the external data storage device, for example, storageon a network or a smart phone or PC, via the communication circuit 14and communication circuit 301.

When carrying out data save in step S141, after communication has beenlost between the camera body 100 and the external data storage device300, a time until it is acceptable to delete the saved data isdesignated, and transmitted in conjunction with information on thisdeletion time. The external data storage device 300 stores the deletiontime that has been input from the camera body 100 in the memory 303.With the example shown in FIG. 10 and FIG. 11, when saved data isreceived, a timeout until restoration in the case where communicationhas been lost is made 30 seconds, and so the deletion time is made atime that is greater than this timeout time (30 seconds) (for example,40 seconds). If less than the timeout time is designated, a conditionwill arise where the file will not exist even with restoration. In acase where processing time for combination is long, then it ispreferable to change the deletion time in accordance with the order ofraw data used in combination processing etc. in consideration of thislength of time.

Once data save has been carried out in step S141, it is determinedwhether or not it is HDR combination (S151), and if the result of thisdetermination is HDR combination RAW acquisition and expansion iscarried out (S153). Once RAW acquisition and expansion has been carriedout it is next determined whether or not there is a failure (S154). Whencommunicating with the external data storage device 300, compared to thecase of carrying out data saved to the storage medium 131 within thecamera body 100, it is more likely that communication errors will arise.With this embodiment, therefore, whether a communication error hasoccurred is determined using whether or not RAW acquisition andexpansion has been successful.

If the determination as to whether or not there has been a failure instep S154 is that at the time of RAW acquisition it was not possible tocommunicate with the external data storage device 300 saved data wastransmitted to, there is a wait for a specified time (for example, 30seconds), and if a communication path cannot be established a failure isdetermined.

If the result of determination in step S154 was that there was afailure, error display is carried out (S183). In this step, characters,icons or the like representing that there has been a failure incombination are displayed on the display panel 135, and the fact thatthere has been a failure in communication is conveyed to thephotographer. Once error display has been carried out the originatingflow is returned to. On the other hand, if the result of detection instep S154 is that there is not an error, processing advances to stepS155 and, similarly to the first embodiment, HDR combination is carriedout.

If focus stacking has been determined in step S161, then similarly tostep S154, whether or not there has been a failure is determined basedon whether a timeout time has elapsed in a state where it is notpossible to establish a communication path, and in the event of afailure error display is carried out (S183), and once error display hasbeen carried out the originating flow is returned to. On the other hand,if the result of detection in step S164 is that there is not an error,processing advances to step S165 and, similarly to the first embodiment,focus stacking is carried out.

If super resolution combination has been determined in step S171, thensimilarly to step S154, whether or not there has been a failure isdetermined based on whether a timeout time has elapsed in a state whereit is not possible to establish a communication path, and in the eventof a failure error display is carried out (S183), and once error displayhas been carried out the originating flow is returned to. On the otherhand, if the result of detection in step S174 is that there is not anerror, processing advances to step S175 and, similarly to the firstembodiment, super resolution combination is carried out.

In this way, in the flow for shooting and image processing on thisembodiments, similarly to the first embodiment, since data save iscarried out for image combination, it is possible to acquire image dataof large capacity even if the capacity of the SDRAM 127 is small. It istherefore possible to improve image quality of a combined image.

Next, operation of data save for the external data storage device 300will be described using FIG. 11. If execution of a program based on mainflow for the external data storage device shown in FIG. 11 is started,it is first determined whether or not communication is possible (S201).Here, whether or not it is a state where communication is possible ischecked. If there is wireless communication or wired communication, itis determined whether or not the communication path has beenestablished. Even in a case where the memory 303 is arranged inside thesame device, this method can be adopted, and in such a case it ischecked whether or not camera functions are active.

If the result of determination in step S201 is a state in whichcommunication is possible, it is next determined whether or not data isreceived (S203). Here it is determined whether or not it was possible toreceive data from the camera body 100. In the case where the memory iswithin the same device, it is determined whether or not there has been adata save request.

If the result of determination in step S203 is that data has beenreceived, next save data is received (S205). Once data receipt has beencarried out, next file storage is carried out (S207). Here, saved datais stored in the memory 303 within the external data storage device 300.Any file name may be used as long as it is not duplicated.

Once file storage is being carried out, it is next determined whether ornot there is a deletion time (S209). In the data save step of S141 inFIG. 10, a deletion time determined in accordance with timeout timeuntil restoration in the case where communication is lost (40 secondswith the example of FIG. 10) is transmitted from the camera body 100 tothe external data storage device 300. In this step it is determinedwhether or not a deletion time has been transmitted from the camera body100. If the result of determination in step S209 is that there is adeletion time, the deletion time is stored (S211).

If the result of determination in step S203 is not data receipt, it isdetermined whether or not there was a data request (S213). In the eventthat the camera body 100 uses data that has been saved to the memory 303of the external data storage device 300 when carrying out imagecombination, a data request is transmitted to the external data storagedevice 300. In this step it is determined whether or not there is thisdata request signal.

If the result of determination in step S213 is that that was a datarequest, data transfer is carried out (S215). Here, RAW data forcombination that is the subject of the data request is read out, andtransmitted to the camera body 100.

Once data transfer has been carried out, file deletion is carried out(S217). Here, the RAW data for combination that has been read out andtransmitted to the camera body 100 is deleted from the memory 303. Iffile deletion has been carried out, the deletion time that was outputaccompanying the RAW data for combination is deleted (S219).

If the result of determination in step S201 is not a state wherecommunication is possible, the deletion time is checked (S221). Since adeletion time is stored in accordance with a number of saved data, it isconfirmed, for each deletion time, whether or not that time has elapsedsince entering the state where communication was not possible.

Once the deletion time has been checked, it is determined whether or nota designated deletion time has elapsed (S223). In this step, it isdetermined whether or not the designated deletion time has elapsed sinceentering the state where communication was not possible.

If the result of determination in step S223 is that the designated timehas elapsed, file deletion is carried out (S225). Since the designatedtime has elapsed, there is a state where communication is not possible,and all RAW data for combination that has been stored in the memory 303is deleted. Once file deletion has been carried out in step S225, nextthe deletion time is deleted (S227).

If the deletion time has been deleted in step S227, if the result ofdetermination in step S223 is that the designated time has elapsed, ifthe deletion time has been deleted in step S219, if the result ofdetermination in step S213 was that there was no data request, if theresult of determination in step S209 was that there was no deletiontime, or if a deletion time has been stored in step S211, it isdetermined whether or not processing is complete (S229). As completionconditions there are the user instructing completion using any operationmember etc., or the power supply of an external device being turned offor restarted etc. If the result of this determination is not completion,processing returns to step S201 and operation continues, while if theresult of determination is completion operation of the main flow iscompleted.

In this way, with the second embodiment, since it is not possible tocontrol the file system of the memory 303 at the camera body 100 side,in the event that the link to the external data storage device 300 islost RAW data for combination within the external data storage device300 is automatically deleted after a specified time (after the deletiontime).

For example, continuous shooting of a plurality of images (for example 8images) is carried out in the camera body 100, some (4) of these takenimages are stored in SDRAM 127 within the camera body 100, and theremaining data (for the remaining 4 images) is saved to the memory 303within the external data storage device 300. At the time of transmittingsaved data from the camera body 100 to the external data storage device300, a deletion time is also transmitted, and using this deletion time,in the event that communication is terminated due to abnormal close downof the camera body 100 etc., the external data storage device 300deletes the saved data.

As has been described above, each of the embodiments of the presentinvention comprises an image data acquisition circuit (for example,image sensor 103) for acquiring image data, a temporary memory (forexample, SDRAM 127) for temporarily storing image data, an imageprocessing circuit (for example, image combining circuit 109 c) forsubjecting the temporarily stored image data to image processing, astorage medium (for example, storage medium 131, memory 303) capable ofstoring at least some saved data of the temporarily stored image data,management information memory (for example, entry 127 w for saved data)for storing management information for management of the saved data, anda control circuit (for example, microcomputer 121) for carrying outmanagement of the saved data using the management information whenstoring the saved data in the storage medium. As a result, even in acase where the imaging device has been closed down abnormally, orsituations such as where communication between storage medium and theimaging device has been interrupted, unnecessary files are notpreserved.

Also, with the embodiments of the present invention, the control circuit(for example, microcomputer 121) stores saved data in a storage mediumwhen carrying out image processing using the image processing circuit,and at that time management information, showing storage position in thestorage medium where the saved data is stored, is stored in managementinformation memory (for example entry 127 w for saved data in the SDRAM127) , but management information is not stored in the storage medium(for example, storage medium 131) (refer, for example, to FIG. 8).Specifically, there are no administrative files in the storage medium(for example, storage medium 131, memory 303).

Also, in the embodiments of the present invention, the imaging devicehas a communication circuit (for example, communication circuit 14) fortransmitting saved data from temporary memory (for example, SDRAM 127)to storage medium (for example, memory 303), and together with saveddata, deletion time information (for example, S141 in FIG. 10) istransmitted to a control circuit (for example, control circuit 305) atthe storage medium side via the communication circuit. As a result, whenthe storage device is external to the imaging device, even in a casewhere the imaging device has been closed down abnormally, or insituations such as where connection or communication between the storagemedium and the imaging device has been interrupted, unnecessary filesare not preserved.

Also, with the embodiments of the present invention the control circuit(for example, control circuit 305) deletes saved data that has beentemporarily stored in the storage medium (for example, memory 303) afterthe lapse of the deletion time (for example, S225 in FIG. 11). As aresult, even in a case where the imaging device has been closed downabnormally, or in situations such as where connection or communicationbetween the storage medium and the imaging device has been interrupted,since files are deleted they are not preserved.

Also, with the embodiments of the present invention, the image datastorage device (for example, image data storage device 300) stores imagedata that has been output from the imaging device (for example, camerabody 100). There is also provided a data acquisition circuit foracquiring saved data together with deletion time via a communicationcircuit (for example, communication circuit 301), a storage medium thatcan store the saved data (for example, memory 303), a managementinformation memory for storing deletion time (for example, memory 303,or may be separate from the memory 303) , and a control circuit (forexample, control circuit 305) for deleting saved data in the event thatthe deletion time has elapsed since a communication path via thecommunications circuit was lost. As a result, even in a case where theimaging device has been closed down abnormally, or in situations such aswhere connection or communication between the storage medium and theimaging device has been interrupted, unnecessary files are not preservedin the image data storage device.

Also, each of the embodiments of the present invention comprises animaging circuit (for example, image sensor 103) for forming a subjectimage and acquiring image data, a memory (for example, SDRAM 127) fortemporarily storing image data, an image combining circuit (for example,image combining circuit 109 c) for combining a plurality of image dataand generating combined image data, a storage control circuit (forexample, microcomputer 121) for controlling data storage, and a controlcircuit (for example, microcomputer 121) for controlling the imagingdevice, the control circuit instructing the imaging circuit so as toconsecutively acquire a plurality of image data (for example, S101-S139in FIG. 4), instructing the storage control circuit to store at leastsome of the plurality of image data using a first storage system (forexample the storage format for the storage medium 131 in FIG. 8), andinstructing the image combining circuit so as to combine a plurality ofimage data, and instructing the storage control circuit to store theimage data that has been generated by the image combining circuit usinga second storage system (for example, the storage format for the SDRAM127 in FIG. 8). As a result, even in a case where the imaging device hasbeen closed down abnormally, or situations such as where connection orcommunication between the storage medium and the imaging device has beeninterrupted, unnecessary files are not preserved. The first storagesystem is a system for storing image data such that it is not seen as afile after at least a specified time has elapsed, while the secondstorage system is a system for storing image data so that it exists as afile after at least a specified time has elapsed.

Also, each of the embodiments of the present invention has non-volatilememory (for example, storage medium 131), and the storage controlcircuit controls data storage to the non-volatile memory such that witha first storage system data is stored to a cluster for data storage ofthe nonvolatile memory and information is not stored as a file in amanagement region (for example, the storage system for the storagemedium 131 of FIG. 8), and with a second storage system data is storedin a cluster for data storage of the non-volatile memory and managementinformation is stored as a file in a management region. As a result,even in a case where the imaging device has been closed down abnormally,or situations such as where connection or communication between thestorage medium and the imaging device has been interrupted, unnecessaryfiles are not preserved.

Also, the embodiments of the present invention have a file deletioncontrol circuit. With the first storage system a deletion time untildeletion after abnormality detection is specified and data is stored(for example, S211 in FIG. 11), while with the second storage systemdata is stored without specifying a deletion time, and the file deletioncontrol circuit deletes a file when a deletion time has elapsed, afteran abnormality has been detected (S225 in FIG. 11). As a result, even ina case where the imaging device has been closed down abnormally, orsituations such as where connection or communication between the storagemedium and the imaging device has been interrupted, unnecessary filesare not preserved.

Also, with each embodiment of the present invention, image data isacquired (for example, S101, S113, S125, S135 in FIG. 4), image data istemporarily stored (for example, S103, S115, S127, S137 in FIG. 4 andstate 6 b in FIG. 6), at least some of the temporarily stored image datais saved as saved data (for example, S141 in FIG. 5 and states 6 c-6 ein FIG. 6), management information for managing the saved data is stored(for example, entry 127 w for saved data in FIG. 8), saved data ismanaged using the management information when storing saved data to astorage medium (for example, FIG. 8), and image processing is carriedout using saved data that has been saved (for example, states 6 c-6 e inFIG. 6). As a result, even in a case where the imaging device has beenclosed down abnormally, or situations such as where connection orcommunication between the storage medium and the imaging device has beeninterrupted, unnecessary files are not preserved.

Also, with the embodiments of the present invention, saved data isacquired together with a deletion time (for example, S205 in FIG. 11),the saved data and the deletion time are stored (for example, S207 andS211 in FIG. 11), and in the event that the deletion time has elapsedsince a communication path for input of saved data was lost, the saveddata is deleted (for example, S223 and S225 in FIG. 11). As a result, insituations such as where communication between the imaging device and animage data storage device has been interrupted, unnecessary files arenot preserved.

Also, with the first embodiment of the present invention, the storagemedium 131 is arranged within the camera body 100, but as long as filemanagement for an external storage medium can be carried out by thecamera body 100, the location of the storage medium 131 may be externalto the camera body. Also, with the second embodiment of the presentinvention, the memory 303 is arranged outside the camera body 100, butas long as it is possible to carry out file management of an externalstorage medium (memory 303) within the camera body 100, the memory 303may be within the camera body. Also, the file management method in whichmanagement information relating to the saved data is not stored, thatwas described in the first embodiment of the present invention, may alsobe carried out with the external data storage device 300 that wasdescribed in the second embodiment of the present invention.

Also, with each of the embodiments of the present invention, shooting iscarried out a plurality of times, and image combination is carried outusing a variety of image data acquired at that time, but it is alsopossible to carryout image combination by dividing single image dataacquired from single shooting into a plurality of sections. Also, imagecombination has been performed using a plurality of RAW data, but imagecombination is not limited to RAW data, and may be performed aftertemporarily converting to JPEG data etc. Also, while image combinationhas been performed using a plurality of image data, processing is notlimited to image combination, and image processing to process aplurality of image data may also be carried out.

Further, with these embodiments, a device for taking pictures has beendescribed using a digital camera, but as a camera it is also possible touse a digital single lens reflex camera or a compact digital camera, ora camera for movie use such as a video camera, and further to have acamera that is incorporated into a mobile phone, a smart phone, a mobileinformation terminal (PDA: Personal Digital Assistant), personalcomputer (PC), tablet type computer, game console etc. In any event, itis possible to adopt the present invention as long as a device carriesout image processing using a plurality of image data.

Also, among the technology that has been described in thisspecification, with respect to control that has been described mainlyusing flowcharts, there are many instances where setting is possibleusing programs, and such programs may be held in a storage medium orstorage section. The manner of storing the programs in the storagemedium or storage section may be to store at the time of manufacture, orby using a distributed storage medium, or they be downloaded via theInternet.

Also, regarding the operation flow in the patent claims, thespecification and the drawings, for the sake of convenience descriptionhas been given using words representing sequence, such as “first” and“next”, but at places where it is not particularly described, this doesnot mean that implementation must be in this order.

The present invention is not limited to these embodiments, andstructural elements may be modified in actual implementation within thescope of the gist of the embodiments. It is also possible form variousinventions by suitably combining the plurality structural elementsdisclosed in the above described embodiments. For example, it ispossible to omit some of the structural elements shown in theembodiments. It is also possible to suitably combine structural elementsfrom different embodiments.

What is claimed is:
 1. An imaging device, comprising: an image dataacquisition circuit for acquiring image data, a temporary storage fortemporarily storing the image data, an image processing circuit forsubjecting the image data that has been temporarily stored to imageprocessing, a storage medium that can store at least some saved data ofthe image data that has been temporarily stored, a managementinformation storage for storing management information for managing thesave data, and a control circuit for, when the saved data is stored inthe storage medium, carrying out management of the saved data using themanagement information.
 2. The imaging device of claim 1, wherein: thecontrol circuit, when carrying out image processing using the imageprocessing circuit, stores the saved data in the storage medium, and atthat time stores management information, showing the storage location atwhich the same data has been stored in the storage medium, in amanagement information storage, but does not store the managementinformation in the storage medium.
 3. The imaging device of claim 1,further comprising: a communication circuit for transmitting the saveddata from the temporary storage to the storage medium, and whereintogether with the saved data, deletion time information is transmittedvia the communication circuit to the control circuit at the storagemedium side.
 4. The imaging device of claim 3, wherein: the controlcircuit deletes the saved data that has been temporarily stored in thestorage medium after the deletion time has elapsed.
 5. An imagingdevice, comprising: an imaging circuit for acquiring image data byforming an image of a subject; a storage for temporarily storing theimage data; an image combination circuit for combining a plurality ofimage data and generating combined image data; a storage control circuitfor controlling data storage; and a control circuit for controlling theimaging device, wherein the control circuit instructs the imagingcircuit to continuously acquire a plurality of image data, instructs thestorage control circuit to store at least some of the plurality of imagedata using a first storage system, instructs the image combinationcircuit to combine the plurality of image data, and instructs thestorage control circuit to store the image data that has been combinedby the image combination circuit using a second storage system.
 6. Theimaging device of claim 5, wherein: with the first storage system imagedata is stored so as not to be shown as a file after at least aspecified time has elapsed, and with the second storage system imagedata is stored so as to exist as a file after at least a specified timehas elapsed.
 7. The imaging device of claim 5, further comprising: anon-volatile memory, and wherein the storage control circuit controlsdata storage for the nonvolatile memory, with the first storage systemdata is stored in a cluster for data storage of the nonvolatile memory,and information is not stored as files in the management region, andwith the second storage system data is stored in a cluster for datastorage of the nonvolatile memory, and management information is storedas files in the management region.
 8. The imaging device of claim 5,further comprising: a file deletion control circuit, wherein with thefirst storage system, data is stored with designation of a timeout timeuntil file deletion after abnormality detection, with the second storagesystem data is stored without designating the timeout time, and the filedeletion control circuit deletes a file when the timeout time elapses,after an abnormality has been detected.
 9. An imaging method comprising:acquiring image data, temporarily storing the image data, saving atleast some of the image data that has been temporarily stored as savedata, storing management information for managing the saved data, whenthe saved data is stored in the storage medium, carrying out managementof the saved data using the management information, and carrying outimage processing using the saved data that has been saved.